Falls are the leading cause of fatal and nonfatal injuries to elders in the modern society. For elders who live alone and independently, about 50% of the falls occur within their own homes, thus timely and automatic detection of falls has long been the research goal in the assistive living community. Various techniques ranging from wearable sensor-based, ambient device-based to computer vision based solutions have been proposed and studied. Wearable sensor-based approaches were among the first techniques developed for fall detection. Since Lord and Colvin proposed an accelerometer-based approach in 1991, numerous kinds of sensors have been explored for fall detection in the past few decades, ranging from gyroscopers, barometric pressure sensors, RFID, to the sensor-rich smart phones. These systems can only work when sensors are worn by the user. However, the always-on-body requirement makes the subject difficult to comply with, especially for the elders at home. Ambient device-based approaches try to make use of ambient information caused by falls to detect the risky activity. The ambient information being used includes audio noise, floor vibration, and infrared sensing data. In these systems, dedicated devices need to be implanted in the environment. However, the other sources of pressure or sound around the subject in the environment account for a large proportion of false alarms. Computer vision-based approaches use cameras installed in the monitoring environment to either capture images or video sequences for scene recognition. Although the recent advances in infra-red LED and depth camera like Microsoft Kinect, have enlarged its application scope (e.g., independent of illumination of lights and can work even in a dark room), the privacy intrusions, inherent requirement for line of sight and intensive computation for real-time processing are still open issues that need to be addressed in the future.
Due to the limitations of the above-mentioned fall detection solutions, very few fall detection systems have been widely deployed in real home settings so far. In recent years, the rapid development in wireless techniques has stimulated the research in studying the relationship between the wireless signal and human activities. In particular, the recently exposed physical layer Channel State Information (CSI) on commercial WiFi devices reveals multipath channel features at the granularity of OFDM subcarriers, which is much finer-grained than the traditional MAC layer RSS (Received Signal Strength). By exploiting the amplitude and phase information of CSI across the OFDM subcarrier and the diversity of CSI information across multi-antennas in the MIMO system, significant progress has been made in applications in motion detection, lip language and gesture recognition, vital sign monitoring and activity recognition. The rationale behind all these research efforts is that different human activities can cause different signal change patterns, and activities can be recognized in real-time by mapping the observed signal change patterns to different human activities.
In the prior art, there is a technology using WiFi commodity devices to detect fall. However, it makes two assumptions: (1) the subject can only perform four kinds of predefined activities (i.e., walk, sit, stand up, fall); (2) Activities can not be performed continuously. For example, the subject should stand up and stand for a while, and then walk. Wifall proposed by C. Han, K. Wu, Y. Wang and L. M. Ni in “Wifall: Device-free fall detection by wireless networks” is an example. Both assumptions are not realistic in real home settings. Therefore, in the present invention, the inventors intend to remove both assumptions to detect the fall in the real settings, i.e., various daily activities are performed naturally and continuously.
In order to automatically detect falls in real-time with WiFi signals in the real settings, there are several challenges that must be addressed. Firstly, how the fall and other human activities affect the amplitude and phase information of CSI? Are there any specific features in the CSI of WiFi signal streams which can characterize the fall and other human activities? Secondly, as activities are performed continuously, the boundary of the WiFi signal of subsequent activities is not given. How to automatically and accurately segment the corresponding fall and other activities in the continuously captured WiFi wireless signal streams? Thirdly, as there are countless daily activities, from the perspective of activities recognition, the problem space is infinite. Even if the activities are segmented out, differentiating the fall from all the other daily activities is like searching a solution in an infinite problem space, which is also challenging.
The inventors observe that the phase difference over two antennas exhibits interesting characteristics in the presence of fall and other human activities. Based on this observation, the inventors proposed a transition-based segmentation method leveraging the variance of phase difference over a pair of receiver antennas as a salient feature to automatically segment all the fall and fall-like activities in the continuously captured WiFi wireless signal streams. Then the inventors extracted features from both the amplitude and phase information of CSI to separate the fall from the fall-like activities. In addition to verifying that the phase difference is a more sensitive base signal than the amplitude of CSI, the inventors also observed that the fall and fall-like activities are ended with a sharp power profile decline in the time-frequency domain. Based on these two insights, the inventors design and implement inventors' real-time fall detector, called RT-Fall.
The main contributions of the present invention are as follows:
1) Deal with fall detection problem with commodity WiFi devices in the real settings, i.e., detect the fall in the condition that countless daily activities are performed naturally and continuously.
2) Identify the phase difference of CSI as a better base signal than amplitude for activity segmentation and fall detection. By studying the relationship between different human activities and the variance of phase difference, the inventors demonstrate its effectiveness as a base signal to segment the fall and fall-like activities in the continuously captured signal streams.
3) The inventors found the sharp power profile decline pattern of the fall in the time-frequency domain and further exploited the complementary characteristics of falls in the time and frequency domain for accurate fall segmentation/detection.
4) The inventors design and implement the real-time activity segmentation and fall detection system, RT-Fall on commodity WiFi devices, with only one antenna at the transmitter side, and two antennas at the receiver side. Experimental results demonstrate that RT-Fall can accurately segment fall and fall-like activities in WiFi wireless signal streams, and has a better fall detection performance than WiFall, the highest level fall detector currently available.